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What  is  NAVAIR? 


•  NAVAIR  is  the  Naval  Air  Systems  Command 

•  Develop,  acquire,  and  support  the  aircraft  and 
related  weapons  systems  used  by  U.S.  Navy  and 
Marine  Corps 

•  Our  goal  is  to  provide  the  fleet  with  quality 
products  that  are  both  affordable  and  available 
when  most  needed 

•  Our  support  extends  across  the  entire  life  span  of  a 
product,  including  all  upgrades  and  modifications 
to  that  product 
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Where  is  NAVAIR? 


China  Lake 

WEAPONS 
DIVISION 


Pt  Mugu 

WEAPONS 

DIVISION 


NADEP  DEPOT 

North 
Island 

NATEC  DEPOT 

NAVAIR  Headquarters 
O  Acquisition/Test/Development  Centers 
A  Naval  Aviation  Depots _ 


Orlando 

TRAINING 

SYSTEMS 

DIVISION 


Lakehurst 

ALRE  -  SUPPORT  EQ 
AIRCRAFT  DIVISION 


Patuxent  River 

NAVAIRHQ,  PEOs 
AIRCRAFT  DIVISION 


Cherry 
Point 

NADEP  DEPOT 


Jacksonville 

NADEP  DEPOT 


Slides 


Approved  for  Public  Release 


NAV,^AI  R 


NAVAIR  PI  History 


2011 

2010 

2009 

2008 

2007 

2006 

2005 

2004 

2003 

2002 

2001 

2000 

1999 

1998 

1997 

1996 

1995 

1994 

1993 


Change  Management 


Process  Modeling 


TPI Launches 


TPI  Research 


CMMI 


TSP  Launches 


PSP  classes 


CMM 


Process  Improvement  Phase 


Team  Process 
Based  Systems+ 


Team  Process 
Based  Systems 

Model  Based  Syste^^^ 
Team  Process 
Based  Software 

Personal  Process 
Based  Software 

Model  Based  Software 
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TPI  Implementation 
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Models  and  Processes 


Capability  Maturity  Models: 

Reference  for  organizations 
buiiding  process  capability 


Team  Processes: 

Processes  for  teams 
building  quality  products 
on  cost  and  schedule 


in/iuii 
I  I  I 


iiuiun  wuivii 

III  I  I  I 


Personal  Processes; 

Processes  used  to  train 
individual  skill  and 
discipline 
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Key  Team  Process  Framework 
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Team  Process  Elements 


Scripts 


Document  the  process  entry  criteria,  phases/ 
steps,  and  exit  criteria.  The  purpose  is  to  guide 
you  as  you  use  the  process. 


M6aSUr6S  Measure  the  process  and  the  product.  They 

provide  insight  into  how  the  process  is  working 
and  the  status  of  the  work. 


Forms, 

Logs, 

Charts 

(paperless) 


Provide  a  convenient  and  consistent 
framework  for  gathering,  retaining,  viewing 

data 


Standards  Provide  consistent  definitions  that  guide 

the  work  and  gathering  of  data. 


Slide  10 


Approved  for  Public  Release 


NAV,^AI  R 


Team  Measures  and  Metrics 


•  Each  team  member  gathers 
four  basic  measures 

-  Times 

-  Sizes 

-  Mistakes 

-  Task  completion  dates 


Charts  and  tables  of  project 
metrics  are  available 
(updated  in  real  time) 


Direct  Hours 


Earned  Value 


Tasks  in  Progress 


many  more... 
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NAVAIR  TPI 


•  Success  of  software  teams  using  TSP  led  their 
organizations  to  ask  for  same  performance  on 
other  teams 

-  Worked  with  the  SEI  to  develop  approach 

-  Based  on  same  TSP  fundamental  principles 


•  NAVAIR  approach  has  become  TPI  for  all  teams 

-  Teams  plan  all  work  from  first  launch  forward 

-  Work  is  based  on  all  products  and  services  defined  in 
process  modeling 

-  PSP  for  Engineers  training  planned  as  part  of  project  if 
appropriate 
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Evolution  of  the  TPI  Approach 


•  Training  has  become  just-in-time 

•  Teams  immediately  begin  to  define  quality  for 
themselves 

-  Log  nnistakes  during  first  cycle 

-  First  postmortem  analysis  of  mistakes  leads  to 
identification  of  mistake  types 

-  Second  launch  will  begin  the  application  of  mistake  types 

•  Explicit  process  modeling  techniques  added  prior  to 
launch 

-  Better  supports  team's  unique  measurement  framework 

-  Enables  team  ability  to  establish  firm  foothold  on 
planning  and  tracking 


Slide  13 


Approved  for  Public  Release 


NAV,^AI  R 


Process  Modeling 
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Process  Modeling 


•  Method  for  describing  processes 

-  Existing  "as  is"  processes 

-  Desired  "to  be"  processes 


BEFORE 


AFTER 
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Process  Modeling 


Each  field  captures  certain  aspects  of  the 
process  activity 


Activity  Name 

Activity  Purpose 

Activity  Constraints 

VERB-NOUN 

Why? 

Conditions? 

Tailoring 

Participants 

Measurements 

Exceptions? 

Who? 

What? 

■ 


The  nouns 
and  verbs 
identified 
here 

become  key 
in  the 

definition  of 
the  life  cycle 
models 
unique  to 
each  team 
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Scripted 
Process 
Resu  Its 

•  Given  to  team  for 
peer  review  prior  to 
launch 

•  Reviewed  by  team  in 
launch  for  quality 
removal  potential 

•  Maintainable  process 
artifacts  post  launch 


Process  Name:  Perform  Ground  Testing 


Purpose 


Verify  performance  of  system  undertestinaircraftinsafe  ground 
environment 


Controls 


Constraint  aircralt  available 

Testplan 

TECT 


Tailoring 


a^Bja^testing 


Participants 


Test  Engineers 

MXsupport 

Aircrew 

Contractors;  Bell.  NG) 
PMA 


Measurements 


Test  coordinatorspreadsheets 

data  base  metrics 


TnpuT^ 


^^tproceou 


W 


rest  procedures  (from  CM.i 

Aircraft  mod  package 
Aircraft  configuration 
GSELicenses  andCerts 
Fly  Me 

Weekly  airgaft  schedule 


II  y  iivi  141 
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BwaiPf!>iiigBnri 


Ensure  test  equipment  available 
Ensure  aircraftisavailable 
Coordinate  ground  turn  if  required 
Coordinate  capture  carry  artide  if  required 
Coordinate  with  test  coordinator 


General 


l/j€8cn 


step 


Activities 


lescnption  z=:3  s 


Check  in  with 

QA 

maintenance 


Perform  aircralt 

setup  ground 
test  procedures 


rBu 


TBIT 


Run 

procedures 

Redline 

procedures 


TBIT 


Verity  repeatable  conditions 

Check  out  QA  and  maintenanc 
.Create  TAR 


Kesuiis  exist  for  eacn  test  point 

TAR  created 


Performed  Test 

TAR# 

Redlined  procedures 
Notes  and  data  for  SARs 
Test  event  data 


Outputs 
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TPI  And  Beyond! 
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Just-in-Time  TPI  Training 

Learning  Doing 
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TPI  Pluses  &  Minuses 

+A  detailed  plan! 

+ Ability  to  track  progress  (weekly) 

+  Improved  estimating  (over  cycles) 

-  No  mature  processes 

-"Where  do  we  put  defect  removal  phases?" 

-  No  defect  type  standards 

-"How  do  we  populate  Review  Checklists?" 

-  No  quality  planning 

-"Will  our  plan  produce  a  good  product?" 
-No  quality  indicators  (e.q.,  A/FR) 
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CMML  TSiCMMiRelationship 


T  TTP 

I 


(Sys  Test) 


P  PTP 
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TPI 

TRP 

(Rqnits) 

TSP 

(SAV) 

TTP 

(Sys  Test) 

•  •• 

•  •• 

TxP 

PRP 

PSP 

PTP 

PxP 

SS  TPI  is  Only  a  Waypoint 

•  TPI  teams  will  hit  a  glass  ceiling 

•  TPI  teams  need  to  evolve  to  achieve  TSP- 
like  performance  (become  a  TxP  team) 

•  What  else  does  a  TPI  team  have  to  do  in 
order  to  become  a  TxP  team? 

•  What  does  a  TSP  team  do? 
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What  Does  a  TSP  Team  Do? 


(Plan  the  Work) 


(Work  the  Plan) 


(Analyze  the  Data) 


Launch 


Weekly  Meetings  and  Day-to-Day  Actions 


Typical  TSP  Cycle 


Postmortem 
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TxP  Planning  Activities 


From 

Some 

Get 

The 

Time 

To 

Start 

Later 

Last 

Project  and  Management  Objectives  (LAU  1) 
Team  Goals  and  Roles  (LAU  2)  . . 


Project  Strategy  and  Support  (LAU  3) 
Overall  Plan  (LAU  4) 


Planned  sizes  and  rates  used  to  compute  times  (LAU  4) 
Quality  Preparation  (LAU  5) 


a 


Planned  Defects  Injected/Removed  (LAU  5) 

Planned  quality  indicator  values  are  acceptable  (LAU  5) 
Balanced  Plan  (LAU  6) 


Project  Risk  Analysis  (LAU  7) 

Launch  Report  Preparation  (LAU  8) 
Management  Review  (LAU  9) 
Launch  Postmortem  (LAU  10)  - 


Approved  for  Public  Release 
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TxP  Working  Activities 


From  Some  Get 

The  Time  To 

Start  Later  Last 


Logging  time 
Logging  defects 
Tracking  EV 

Using  PROBE  in  Planning  phase 
Entering  actual  sizes  in  Postmortem  phase 
Defining  Defect  Types 
Using  Review  checklists 
Holding  periodic  team  meetings 
Following  an  agenda  during  team  meetings 
Performing/reporting  on  assigned  roles  — 
Reviewing  action  items 
Reviewing  assigned  goals  and  risks 
Maintaining  project  plan  and  workbook  — 
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TxP  Analyzing  Activities 


Evaluate  plan  vs.  actual  schedule  hours 
Evaluate  plan  vs.  actual  component  hours 

Evaluate  plan  vs.  actual  component  sizes  - 

Evaluate  team  performance  vs.  goals  and  quality  plan 

Evaluate  plan  vs.  actual  quality  of  components  - 

Update  planning  data  for  schedule  hours  - 

Update  planning  data  for  lifecycle  time-in-phase  %s  " 

Update  planning  data  for  productivity  rates  - 

Update  planning  data  for  defect  densities - 

Update  planning  data  for  defect  rates  and  yields - 

Update  planning  data  for  quality  indicator  thresholds 


From 

Some 

^lEt 

The 

Time 

To 

Start 

Later 

Last 
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Transitions 


Training  & 
First  Launch 

3-part  TPI 
Training 

'^Process 

Modeling 

'^First  Launch 


Product  Size 
Definition 


v^Define  size 
measures 

Add  Planning 
and  Postmortem 
phases 

'^Begin  use  of 
PROBE 


v^Define  Defect 
Types 

'^Refine  Processes 
with  Defect 
Removal  Phases 


v^Define  Product 
Quality  Indicators 

v^Define  Process 
Quality  Indicators 


Stages 


Planning 

Activities 


Working 

Activities 


Analyzing 

Activities 
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Things  to  Remember 


•  TxP  may  be  applied  to  any  team  that  has  recurring 
work  to  perform 

•  TxP  teams  should  plan  their  work,  work  their  plans, 
and  analyze  their  data  to  improve 

•  This  analysis  gives  them  insight  into  the  quality  of 
their  processes  used  to  produce  their  products  and 
provide  their  services 
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NAVAIR  PRT 


Jeff  Schwalb:  760  939-6226 
Brad  Hodgins:  760  939-0666 
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Backup  Slides 
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NAVAIR  Team  Performance 
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NAVAIRTeam  Data  Profiles 

FYlO-FYll 


Num  of  Teams  19 

Num  of  Teams  (by  type  of  work  performed) 

Tactical/Embedded  Software  Dev  12 
Desktop  Software  Dev  6 

Systems  Integration  1 


Min 

2 

4 


Avg  Max 
6  12 
9  18 
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NAVAIR  Teams 

Schedule 


Schedule  Deviation  Comparison 
NAVAIR  Teams  versus  Industry  Standards 
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*2009  Chaos  Report.  Standish  Group  (3X+  organcations  surveyed) 


Slide  34 


Approved  for  Public  Release 


nav,^ai  r 


NAVAIR  Teams 

Effort  Performance 


Effort  Deviation  Comparison 
NAVAIR  Teams  versus  Industry  Standards 
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&0% 
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-20% 


-40% 
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33% 
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-22% 
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-23% 


•  13% 


-28% 
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*2009  Chaos  Report.  Stan  dish  Group  (S^OO-t-  organisations  surveyed) 
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NAVAIR  Teams 

Quality  in  Test  Time 


60% 


50% 


A0% 


3C'% 


20% 


10% 


0% 


126% 


Test  Percentage  Comparison 
NAVAIR  Teams  versus  Industry  Standards 


T0%  29% 


31% 


M/A 


N.'A 


N/A 


19% 


15% 


N/A  N/A 


M/A 


N'  “O'  V*  N**  N'-*  N'  ‘C’ 


•So  .A 


12% 


^«>  ^e  ^e>  ^  ^e>  ^ 

*  --Gnes.  Caoefs.  ed.  Tutorial:  Proorammino  Procuctivit;:  Issues  for  the  Eighties.  2nci  Ed  ..  1935 
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NAVAIR  Teams 

Cost  of  Improvement 


Process  Improvement  /  Development  Cost  Ratio  Comparison 
NAVAIR  Teams  versus  Industry  Standards 


*  Average  percentage  of  on-line  ROI  Process  Improvement  case  studies 
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Trademarks  and  Service  Marks 


•The  following  are  service  marks  of  Carnegie  Mellon  University. 

-  Team  Software  Process^'^ 

-  TSPSM 

-  Personal  Software  Process^'^ 

-  PSPSM 


•The  following  are  registered  trademarks  of  Carnegie  Mellon 
University. 

-  Capability  Maturity  Model® 

-  CMM® 

-  Capability  Maturity  Model  Integration® 

-  CMMI® 

-  CERT® 
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